Towards automatic configuration of big data processing systems

The goal of this research is to develop a generic framework for automatic configuration optimization of big data processing systems. The framework will allow users to specify their performance goals as well as their own metrics. We plan on utilizing system developer provided hints for searching suitable configurations for big data application deployments. Introduction Big data processing systems have dozens of available configuration parameters and system performance crucially depends on tuning several of these parameters. Moreover, the suitable configurations vary depending upon the available resources, workloads and applications for which the system is being used. Today, optimization of these configurations is done manually, possibly requiring several hours of investigation and testing by performance engineers. Performance engineers also need to have detailed knowledge about the big data system under consideration. This makes configuration optimization a tedious and time consuming task. Most of existing works present solutions specific to MapReduce jobs [2,3]. However, to the best of our knowledge the broader research challenge remains open. Preliminary Analysis To understand the variation in performance due to system configurations, we conducted preliminary tests on a multi-node Apache Storm cluster. The setup includes 3 nodes, with the capability to run 96 threads simultaneously. We use the Rolling word count topology of the Intel Storm benchmark [1], with count window size set to 10s and count emit frequency set to 1s. Configuration Possible values Number of workers and acker threads 3-12 (each) Spout, Split Bolt and Count bolt threads 3-64 (each) Table 1 Selected configurations and their values. Following an Experimental Design approach, we use a space filling algorithm to cover the configuration space. Table 1 shows the space of possible configurations considered for each experiment. We have fixed the total number of threads in the topology to 96. We vary the number of threads assigned to each of the topology component while keeping the total number of threads constant. We used the WSP space filling algorithm [4] to generate 77 configurations for our experiments. Figure 1 shows the variation of different performance metrics across different configurations. The per-tuple latency varies between 3ms to 108ms at the median and from 3ms to 577ms at the 99th percentile. In addition, there is a substantial variation in throughput from 10k tuples/s to 117k tuples/s. Thus we observed a variation as high as 2 orders of magnitude in latency and 1 order of magnitude in throughput, given a fixed resource budget. The interplay between parallelism of different Storm components has lead to this drastic variance in performance. The amount of work done by each component of the topology and its processing latency, the number of executor for each bolt/spout and their placement, all play a part in the resulting overall performance. Figure 1: Performance results for selected configurations.